Read this piece on our election site May2015.com.
If you’ve driven your car 20,000 times and never crashed, does that mean you won’t crash if you drive it drunk?
You can’t answer this question with any basis. Without any past data, you just have blind confidence or innate doubt, depending on your disposition. This is the prototypical example of an ‘out-of-sample’ event, used by Nate Silver in his 2012 book The Signal and The Noise. We don’t have any idea how to predict what will happen because we’ve never driven drunk (in this hypothetical).
The sub-prime crisis that crashed the world economy seven years ago was an out-of-sample event: ratings agencies made extremely inaccurate forecasts about whether mortgages would fail because the only data they had was irrelevant. This is exactly the issue general election forecasters in the UK now face.
Like the housing crisis, the rise of the SNP is an ‘out-of-sample’ event.
Throughout the early 2000s American ratings agencies like Moody’s used housing data from the 1980s and 1990s, when house prices barely budged, to predict the likelihood of houses defaulting in an era when prices had suddenly doubled.
The ratings agencies were making forecasts in a world without precedent. So now is May2015, and all the other election forecasters. Not because of the rise of Ukip and the Greens (that’s unusual but third parties have risen before), or the importance of minor nationalist parties (see 1974), but because of one party and one phenomenon: the SNP.
The rise of the SNP
How many seats will the SNP win? That’s the question every election-watcher is asking today after Ashcroft’s latest polls last night.
The predictions we track here on May2015 have now all been updated. The bookies’ forecast for the SNP has edged up by a few seats although Election Forecast’s hasn’t yet, while ours and the Guardian’s were already very high on the SNP.
There are two groups: we and the Guardian think the SNP will win more than 50 seats, the bookies and academics think it will be closer to 40.
But all these nice round numbers mask incredible uncertainty. We are all flying blind. The rise of the SNP is an out-of-sample event: a party hasn’t before gone from winning 20 per cent in one national election to winning 50 per cent in the next.
These nice round numbers mask incredible uncertainty.
The academic models deal with this fact by discounting the polls we’re seeing in Scotland. As the team over at Election Forecast have explained, we can study past elections to see how well the polls typically match up with actual election results. This allows us to say that polls have historically been about 65 per cent ‘accurate’ more than 60 days before polling day.
As we approach the election, the predictive power of polls increases (although they are still only 80 per cent accurate by polling day). New polls are discounted less and matter more in any model.
This idea – of discounting new polls based on historical data – makes sense in general. But it stumbles when we come to Scotland. Doing so means using the recent history of UK-wide elections to water down support for the SNP.
That’s fine if you think the collapse of Labour in Scotland is similar to the dips Labour have suffered before previous elections. But if you think its an event without historical example – which it is – maybe we shouldn’t discount current polls.
This election isn’t impossible to predict, but the rise of the SNP is complicated.
This is why May2015’s model is a ‘nowcast’ rather than a ‘forecast’, and we really offer a projection rather than a prediction. We still call it a prediction because we’re just saying ‘This is what will happen – unless something changes’.
That’s not to criticise forecasts: Election Forecast and Elections Etc are both academic models with more detail, complexity and sophistication than ours. It’s just to point out the difficulty of forecasting the SNP’s fate, and explains why we and the Guardian (whose model is also a nowcast) differ from the academics and bookies.
The upshot of all this isn’t that no one knows. We’re not going to start grandly declaring this election “impossible to predict”. We have a good idea of how the parties will fare in May, thanks to a regular stream of national polls (about 10 a week), periodic batches of Ashcroft seat polls (146 so far), and demographic analysis.
But we have nothing like as much information as US forecasters like Silver, who could plug dozens of state-level polls into his model, and only had two parties to consider. Until more billionaires follow Ashcroft’s lead, predicting British elections will always be guesswork.
Follow May2015.com.
Content from our partners
Related
